Top-1 CORSMAL Challenge 2020 Submission: Filling Mass Estimation Using Multi-modal Observations of Human-Robot Handovers

Human-robot object handover is a key skill for the future of human-robot collaboration. CORSMAL 2020 Challenge focuses on perception part this problem: robot needs to estimate filling mass container held by human. Although there are powerful methods in image processing and audio individually, answering such problem requires data from multiple sensors together. The appearance container, sound filling, depth provide essential information. We propose multi-modal method predict three indicators mass: type, level, capacity. These then combined container. Our obtained Top-1 overall performance among all submissions both public private subsets while showing no evidence overfitting. source code publicly available: github.com/v-iashin/CORSMAL.